542 research outputs found
Linking GloVe with word2vec
The Global Vectors for word representation (GloVe), introduced by Jeffrey
Pennington et al. is reported to be an efficient and effective method for
learning vector representations of words. State-of-the-art performance is also
provided by skip-gram with negative-sampling (SGNS) implemented in the word2vec
tool. In this note, we explain the similarities between the training objectives
of the two models, and show that the objective of SGNS is similar to the
objective of a specialized form of GloVe, though their cost functions are
defined differently.Comment: 5 pages, 2 figure
An Enhanced Multiway Sorting Network Based on n-Sorters
Merging-based sorting networks are an important family of sorting networks.
Most merge sorting networks are based on 2-way or multi-way merging algorithms
using 2-sorters as basic building blocks. An alternative is to use n-sorters,
instead of 2-sorters, as the basic building blocks so as to greatly reduce the
number of sorters as well as the latency. Based on a modified Leighton's
columnsort algorithm, an n-way merging algorithm, referred to as SS-Mk, that
uses n-sorters as basic building blocks was proposed. In this work, we first
propose a new multiway merging algorithm with n-sorters as basic building
blocks that merges n sorted lists of m values each in 1 + ceil(m/2) stages (n
<= m). Based on our merging algorithm, we also propose a sorting algorithm,
which requires O(N log2 N) basic sorters to sort N inputs. While the asymptotic
complexity (in terms of the required number of sorters) of our sorting
algorithm is the same as the SS-Mk, for wide ranges of N, our algorithm
requires fewer sorters than the SS-Mk. Finally, we consider a binary sorting
network, where the basic sorter is implemented in threshold logic and scales
linearly with the number of inputs, and compare the complexity in terms of the
required number of gates. For wide ranges of N, our algorithm requires fewer
gates than the SS-Mk.Comment: 13 pages, 14 figure
Relation Structure-Aware Heterogeneous Information Network Embedding
Heterogeneous information network (HIN) embedding aims to embed multiple
types of nodes into a low-dimensional space. Although most existing HIN
embedding methods consider heterogeneous relations in HINs, they usually employ
one single model for all relations without distinction, which inevitably
restricts the capability of network embedding. In this paper, we take the
structural characteristics of heterogeneous relations into consideration and
propose a novel Relation structure-aware Heterogeneous Information Network
Embedding model (RHINE). By exploring the real-world networks with thorough
mathematical analysis, we present two structure-related measures which can
consistently distinguish heterogeneous relations into two categories:
Affiliation Relations (ARs) and Interaction Relations (IRs). To respect the
distinctive characteristics of relations, in our RHINE, we propose different
models specifically tailored to handle ARs and IRs, which can better capture
the structures and semantics of the networks. At last, we combine and optimize
these models in a unified and elegant manner. Extensive experiments on three
real-world datasets demonstrate that our model significantly outperforms the
state-of-the-art methods in various tasks, including node clustering, link
prediction, and node classification
Phone-aware Neural Language Identification
Pure acoustic neural models, particularly the LSTM-RNN model, have shown
great potential in language identification (LID). However, the phonetic
information has been largely overlooked by most of existing neural LID models,
although this information has been used in the conventional phonetic LID
systems with a great success. We present a phone-aware neural LID architecture,
which is a deep LSTM-RNN LID system but accepts output from an RNN-based ASR
system. By utilizing the phonetic knowledge, the LID performance can be
significantly improved. Interestingly, even if the test language is not
involved in the ASR training, the phonetic knowledge still presents a large
contribution. Our experiments conducted on four languages within the Babel
corpus demonstrated that the phone-aware approach is highly effective.Comment: arXiv admin note: text overlap with arXiv:1705.0315
Deep Speaker Feature Learning for Text-independent Speaker Verification
Recently deep neural networks (DNNs) have been used to learn speaker
features. However, the quality of the learned features is not sufficiently
good, so a complex back-end model, either neural or probabilistic, has to be
used to address the residual uncertainty when applied to speaker verification,
just as with raw features. This paper presents a convolutional time-delay deep
neural network structure (CT-DNN) for speaker feature learning. Our
experimental results on the Fisher database demonstrated that this CT-DNN can
produce high-quality speaker features: even with a single feature (0.3 seconds
including the context), the EER can be as low as 7.68%. This effectively
confirmed that the speaker trait is largely a deterministic short-time property
rather than a long-time distributional pattern, and therefore can be extracted
from just dozens of frames.Comment: deep neural networks, speaker verification, speaker featur
- …